CUM: An Efficient Framework for Mining Concept Units

Web is the most important repository of different kinds of media such as text, sound, video, images etc. Web mining is the process of applying data mining techniques to automatically discover knowledge from such a diverse, sheer size data so that it can be more easily browsed, organized, and catalogued with minimal human intervention. A web site usually contains a large number of concept entities, each consisting of one or more web pages connected by hyperlinks. A large portion of web search activities aims to locate a set of concept entities relevant to the user query. The web unit mining problem is proposed to discover the concept entities and classify these concept entities into categories. Web page classification mainly assigns one or more concept labels to every web page based on its own content without considering other neighbouring web pages. The existing iterative Web Unit Mining (iWUM) algorithms create more than one web unit (incomplete web units) from a single concept entity. In this paper, we propose a novel non-iterative web unit mining algorithm, Concept Unit Mining (CUM), which finds the set of web pages forming each web unit, and assigns the web unit a concept label based on the structure of the web pages so as reduce the later classification errors. Our experiments using the WebKB dataset show that the disadvantage of iWUM algorithms are removed and over all accuracy is significantly improved.